Existing integration and messaging systems have problems handling large messages. Incoming messages are read into memory in their entirety, such that when a number of large messages are received a system can crash due to a lack of available memory. Some systems try to prevent these problems by limiting the size of messages that can be processed through a system, but this approach is undesirable to users needing to send messages that may occasionally exceed that limitation.
Another existing approach utilizes in-database persistence and in-memory caching on a hub. Persistence saves enough data for recovery purposes, and caching allows messages to be serialized to a Java Message Service (JMS). This allows JMS to enqueue faster, and allows a JMS dequeue to request the message from a cache without having to redo expensive operations like deserialization, decryption, and XML parsing. The problem still exists in that it is necessary to read an entire message into memory in order to process the message.